We have been successful in using Bloom Cloud Engine to process several hundred Gb of data quite easily. Processing such datasets fast can be challenging for any software and hardware, however, you can take some steps to get the best performance. Some hints and tips for faster performance are described below. Please use them at your discretion; we are also pleased to discuss them with you.


Disk Performance

Disk performance generally is the most important factor for processing raw scan data on Bloom Cloud Engine. Since Bloom Cloud Engine would perform significant read and write operations on the disk, here is a brief description of each technology.


If you have been given processed data, then disk performance is not as critical as other factors mentioned here.

NVMe SSD

This is the best configuration with a typical disk IO of 1.5-2 Gb/s, and this hardware is now commonly available. The SSD is capable of performing disk IO that typically exceeds 1Gb/s, but in most computer hardware it is connected through a SATA III interface, which limits this speed to ~300Mb/s.


The keyword M.2 commonly associated with NVMe hardware denotes the physical form of the disk. It does not automatically imply that M.2 is also NVMe. On contrary, it is likely that if the memory or the motherboard does not specifically mention NVMe, it will be using the slower SATA interface.

SATAIII SSD

This is second best option at ~300Mb/s and is most commonly available in current hardware systems.


HDD

If the data is on HDD and not easy to temporarily move to a SSD, please see that it is connected through SATA interface. If the HDD happens to be connected with USB, please connect using blue colored USB3 cable and computer port.


Memory and CPU

Faster memory and CPU is the second most important factor for raw data processing.


Windows Software Configuration

If processing raw scandata, the Anti-virus system check and Windows search on the scandata can slow down data processing significantly. Please discuss with your IT department, if they can arrange a system that circumvents this slowdown. In the minimum, the windows search should be disabled for files with extension ".point".



Created with the Standard Edition of HelpNDoc: Create help files for the Qt Help Framework